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Abstract  -  In  this  paper  we  describe  a  software 
framework  to  enable  heterogeneous,  distributed  data 
fusion  of  disparate  information  sources.  The  frame¬ 
work  is  agent-based  and  consists  of  three  main  el¬ 
ements.  The  first  is  a  generalization  of  the  tar¬ 
get  state  to  a  container  of  arbitrary,  uncertain  at¬ 
tributes.  The  structure  of  this  estimate  can  vary  both 
across  time  and  across  different  nodes  in  the  same 
network.  The  second  is  the  development  of  compos- 
able  process  and  observation  models.  These  make  it 
possible  to  dynamically  change  the  models  at  runtime 
to  fit  the  current  target  state  estimate.  Finally, 
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1  Introduction 

A  key  enabling  technology  for  network  centric  warfare 
is  a  sensor  network.  The  network  is  composed  of  a  set 
of  fusion  nodes.  Each  node  has  the  ability  to  sense  its 
environment,  to  communicate  with  other  nodes,  and 
to  be  able  to  fuse  the  information  collected  locally  and 
transmitted  from  other  nodes.  The  resulting  network 
is,  in  principle,  scalable,  flexible,  and  robust.  Scalabil¬ 
ity  arises  from  the  fact  that  there  is  no  theoretical  limit 
on  the  overall  size  of  the  network.  The  flexibility  arises 
because  the  capabilities  of  the  network  can  be  changed 
dynamically  at  runtime  by  adding  and  removing  nodes. 
Finally,  the  robustness  arises  because  there  is  no  sin¬ 
gle  point  of  failure:  if  a  fusion  node  fails,  the  network 
continues  operate,  albeit  it  reduced  performance. 

Given  these  advantages,  sensor  networks  have  been 
the  subject  of  intensive  research.  Much  of  this  research 
can  be  categorized  into  three  areas.  The  first  area  con- 
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siders  the  physical  implementation  of  such  networks. 
Research  topics  include  the  development  of  compact, 
low  cost,  and  low  powered  hardware  [1],  new  algo¬ 
rithms  for  power  management  [2],  and  network  proto¬ 
cols  for  ad-hoc  networks  [3] .  The  second  area  addresses 
the  problem  of  developing  services  that  can  run  over 
sensor  networks.  Research  technologies  include  net¬ 
work  distributed  objects,  agent-based  technology  [4], 
Fuselet  Technology  [5]  and  web  services  [6].  The  fi¬ 
nal  area  considers  the  problem  of  fusion  architectures 
and  algorithms  that  will  be  run  in  those  services.  Re¬ 
search  topics  include  the  development  of  data  fusion 
algorithms  to  account  for  double  counting  [7,  8]  and 
the  scheduling  of  broadcasts  to  optimize  information 
content  [9]. 

Although  there  has  been  a  significant  amount  of 
work  to  develop  software  architectures  that  integrate 
the  first  two  research  areas,  relatively  little  work  has 
been  carried  out  into  how  the  third  area  can  be  com¬ 
bined  with  the  other  two.  A  number  of  software  li¬ 
braries,  such  as  Bayes++  [10],  have  been  developed  to 
implement  low-level  algorithms  (such  as  Kalman  and 
particle  filters).  However,  these  do  not  consider  how 
these  algorithms  would  be  implemented  in  the  larger 
context  of  a  dynamic  and  time  varying  system. 

In  this  paper  we  consider  the  problem  of  develop¬ 
ing  a  software  framework  to  implement  a  wide  classes 
of  fusion  algorithms  in  a  generalized  sensor  fusion  net¬ 
work.  In  particular,  the  framework  has  the  following 
characteristics: 

1.  The  target  state  is  modeled  as  a  container  of  ar¬ 
bitrary,  uncertain  attributes.  The  composition  of 
the  target’s  state  can  change  both  through  time 
and  across  the  network. 

2.  The  mathematical  models  used  to  transform  the 
target  state  are  assembled  at  runtime  at  each 
node.  The  models  are  a  function  of  the  struc¬ 
ture  of  the  target  state  and  contextual  information 
such  as  the  node’s  computational  capabilities. 
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3.  To  optimize  communication  between  different 
nodes,  a  publish  and  subscribe  mechanism  is  used. 

4.  Nodes  can  use  code  mobility  to  share  and  upgrade 
software  components. 

5.  The  architecture  imposes  very  few  restrictions  on 
the  assumed  behavior  of  the  data  fusion  algo¬ 
rithms.  Therefore,  almost  any  kind  of  published 
data  fusion  algorithm  can  be  used. 

We  make  two  assumptions: 

1.  There  is  an  agreed  upon  standard  for  the  naming 
and  interpretation  of  each  attribute.  This  stan¬ 
dard  should  be  derived  from  an  ontology. 

2.  Each  attribute  is  a  modeled  as  a  real-valued  scalar 
and  the  uncertainty  associated  with  each  state  can 
be  described  as  a  Gaussian  Mixture  Model. 

The  second  assumption  is  made  to  simplify  the  initial 
design  of  the  framework;  future  revisions  will  relax  this 
assumption. 

The  structure  of  this  paper  is  as  follows.  The  prob¬ 
lem  statement  is  discussed  in  more  detail  in  Section  2. 
An  overview  of  our  framework  is  provided  in  Section  3. 
The  target  state  representation  is  discussed  in  Sec¬ 
tion  4.  The  operations  within  a  single  node  are  de¬ 
scribed  in  Section  5.  The  issue  of  the  interaction  be¬ 
tween  multiple  nodes  is  described  in  Section  6.  An 
example  of  this  architecture  is  discussed  in  Section  7. 
Conclusions  are  drawn  in  Section  8. 

2  Problem  Statement 

The  structure  of  the  network  is  shown  in  Figure  1: 
the  network  is  composed  of  a  set  of  fusion  nodes  and 
communication  links  between  those  nodes.  Each  node 
maintains  its  own  estimate  of  the  Common  Opera¬ 
tional/Tactical  Picture  (COTP).  The  COTP  consists 
of  a  set  of  entities.  Each  entity  is  a  container  of  at¬ 
tributes.  There  are  many  kinds  of  attributes  includ¬ 
ing  kinematics  (e.g.,  position,  velocity),  identification 
(e.g.,  entity  type,  entity  class),  and  sensor  signatures 
(e.g.,  radar  cross  section,  color). 

The  internal  operation  of  each  sensor  is  described 
by  the  event-driven  loop  shown  in  Figure  2.  The  tar¬ 
get  state  is  initialized.  The  node  then  waits  until  it 
receives  an  event.  Some  types  of  events  (such  as  sen¬ 
sor  observations)  provide  sources  of  new  information. 
Other  types  (such  as  timeouts)  are  control  signals  and 
provide  no  new  information.  The  target  state  is  pre¬ 
dicted  forwards  to  the  event  time.  If  the  event  pro¬ 
vides  new  information,  this  information  is  fused  into 
the  target  state.  If  the  event  does  not  provide  any  new 


Figure  1:  The  distributed  data  fusion  network  archi¬ 
tecture. 


Figure  2:  The  generalized  fusion  architecture. 


information,  the  estimated  target  state  is  set  to  the 
predicted  target  state.  In  either  case,  the  loop  returns 
to  its  waiting  condition  again. 

The  way  in  which  this  loop  is  instantiated  on  each 
node  depends  on  four  factors: 

1.  Local  sources  of  information.  Different  fusion 
nodes  have  different  means  of  acquiring  informa¬ 
tion  locally.  Some  nodes  are  equipped  with  sensors 
that  can  directly  measure  various  target  attributes 
with  various  levels  of  fidelity.  Other  nodes  have  in¬ 
direct  sources  of  information.  For  example,  a  node 
could  use  a  terrain  traversability  database  to  con¬ 
strain  the  motion  of  a  target  of  a  particular  class 
type.  Other  types  of  nodes  have  no  local  source  of 
information.  Instead  these  can  act  as  routers  or 
fusion  hubs,  fusing  estimates  from  several  different 
nodes. 

2.  Local  processing  capabilities.  Sensing  nodes 
span  from  small,  unattended  grounds  sensors 
through  UAVs  to  entire  command  centers.  Com¬ 
putational  and  storage  costs  can  vary  by  many 
orders  of  magnitude  and,  as  a  result,  some  types 
of  calculations  simply  cannot  be  carried  out  on 
some  types  of  nodes.  As  an  example,  certain  types 
of  sensor  data  can  be  collected  and  transmitted 
very  easily  but  are  computationally  very  difficult 
to  process1.  Therefore,  some  nodes  might  simple 
broadcast  data  they  collect.  Other  nodes  could 

1  A  proximity  detector  provides  1  bit  of  information  for  “de¬ 
tect”  or  “no-detect” .  However,  if  this  information  is  to  be  used 
to  refine  the  ( x ,  y)  position  of  a  target,  the  calculations  involved 
become  non-trivial. 


Figure  3:  The  structure  of  each  agent. 


Figure  4:  The  communication  topology  for  the  LUS. 
Like  the  data  fusion  network,  this  is  fully  distributed 
and  can  have  an  arbitrary  network  topology. 


process  that  data,  but  with  different  levels  of  fi¬ 
delity. 

3.  Target  state  representations  used.  The  tar¬ 
get  state  representation  is  not  fixed  and  im¬ 
mutable.  It  can  change  over  time  and  across  the 
network  [11].  Some  of  these  choices  are  limited 
by  computational  costs,  some  by  communication 
overhead  and  some  by  security  issues.  For  exam¬ 
ple,  some  attributes  are  privileged  and  can  only 
be  distributed  to  a  subset  of  nodes  in  the  net¬ 
work  [12]. 

4.  Available  bandwidth.  The  bandwidth  is  a 
function  of  many  factors  including  the  hardware 
equipped  on  the  node,  current  environmental  con¬ 
ditions,  and  the  current  sensor  geometry.  Decreas¬ 
ing  bandwidth  causes  a  trade-off  between  the  fi¬ 
delity  of  the  state  estimate  that  is  distributed  and 
the  rate  at  which  it  is  distributed.  As  an  ex¬ 
ample,  the  target  estimate  is  a  Gaussian  mixture 
model  which  consists  of  multiple  modes.  As  the 
bandwidth  decreases,  the  update  rate  can  only  be 
maintained  if  the  distributed  track  state  merges 
modes  and  /  or  deletes  attribute  values. 

Any  software  framework  for  generalized  distributed 
data  fusion  must  address  these  four  factors.  We  pro¬ 
pose  to  use  an  agent-based  architecture. 

3  Overview  of  the  Approach 

Each  sensor  node  is  modeled  as  a  software  agent.  The 
reason  for  using  agents  is  that  they  offer  methods  of 
communication  across  the  network,  lookup  services, 
and  advertisement  capabilities.  The  structure  of  each 
agent  is  shown  in  Figure  3.  The  fusion  loop  from  Fig¬ 
ure  2  is  augmented  by  a  model  repository  and  an  agent 


interface  layer.  The  model  repository  stores  the  math¬ 
ematical  components  used  to  construct  the  different 
types  of  mathematical  models.  The  agent  interface 
layer  mediates  all  inter  agent  communication.  Most 
of  this  work  is  carried  out  through  the  PlatformAgent. 
The  PlatformAgent  represents  the  agent  stored  on  a 
local  platform  and  performs  all  operations  associated 
with  publishing  and  collecting  observation  models,  ob¬ 
servation  data,  and  other  agent  interaction  messages. 

All  agents  coordinate  their  activities  through  the 
distributed  network  of  Look-Up-Services  (LUSs)  illus¬ 
trated  in  Figure  4.  The  LUS  facilitates  coordination 
between  nodes  by  maintaining  a  list  of  available  nodes 
and  their  capabilities.  These  capabilities  are  expressed 
as  a  set  of  advertisements  that  include  what  informa¬ 
tion  a  node  can  provide  (from  its  onboard  sensors  and 
track  estimates),  what  models  it  contains,  and  what 
processing  capabilities  it  has. 

Nodes  communicate  with  one  another  through  a 
publish  and  subscribe  mechanism.  Suppose  a  node  X 
wishes  to  receive  the  set  of  attributes  {Ax}  from  a 
node  Y .  X  passes  its  registration  request  to  Y  and  Y 
only  delivers  {Ax}- 

To  implement  the  agent  services  we  use  the  CoABS 
Grid  [4],  hereafter  referred  to  as  The  Grid.  The  Grid 
was  originally  developed  for  DARPA  as  a  framework 
to  control  large  systems  of  software  agents.  In  partic¬ 
ular,  it  is  optimized  to  aid  communication  and  intelli¬ 
gence  gathering  in  military  domains,  an  environment 
where  bandwidth  availability  and  connected  hardware 
is  constantly  changing.  The  Grid  wraps  Sun’s  Jini  ser¬ 
vices  [13]  and  allows  software  agents  to  register  and  ad¬ 
vertise  capabilities  to  the  LUS.  These  advertisements 
are  customizable  and  change  dynamically  as  an  agent’s 
capabilities  change.  To  support  changing  the  capabil¬ 
ities  of  an  agent,  the  Grid  supports  code  mobility:  one 
agent  can  download  portions  of  the  codebase  of  another 
agent  to  permit  it  to  use  the  new  capabilities.  We  ex¬ 
ploit  this  to  allow  agents  to  share  models,  attributes, 


Figure  5:  The  generalized  estimate  representation. 
The  estimate  consists  of  a  set  of  Attributes,  the  Gaus- 
sianMixtureModel,  and  the  Time. 

and  their  current  states  with  other  agents. 

To  map  these  capabilities  into  the  fusion  algorithms, 
we  describe  the  architecture  in  three  parts:  the  repre¬ 
sentation  of  the  state  estimate,  the  operations  which 
occur  within  a  node,  and  the  operations  that  occur 
between  nodes. 

4  Estimate  Representation 

All  uncertain  quantities  in  the  architecture  are  referred 
to  as  Estimate s.  These  include  the  target  state,  sen¬ 
sor  observations,  and  information  acquired  from  other 
sources  such  as  databases.  Because  the  state  space 
representation  can  vary  for  the  same  target  between 
different  nodes,  each  estimate  must  maintain  the  meta¬ 
data  s(k)  to  specify  the  structure  of  the  estimate.  This 
information  is  contained  in  the  Estimate  class,  illus¬ 
trated  in  Figure  5.  This  aggregates  three  separate 
classes:  a  container  of  Attributes,  the  GaussianMix- 
tureModel  that  specifies  the  probability  distribution 
associated  with  the  estimate,  and  the  Time  when  the 
observation  was  taken. 

Each  Attribute  instance  stores  a  specific  value  about 
an  entity.  As  explained  above,  we  assume  that 
there  is  an  agreed  upon  standard  for  the  naming 
and  interpretation  of  each  attribute.  These  standards 
should  be  defined  by  an  ontology.  Dorion’s  Com¬ 
mand  and  Control  Information  Exchange  Data  Model 
(C2IEDM)  [14],  for  example,  considers  the  battlespace 
to  be  populated  by  a  set  of  OBJECT-ITEMs.  At¬ 
tributes  associated  with  each  OBJECT-ITEM  include 
the  OBJECT-ITEM-LOCATION  (where  is  it?),  the 
OBJECT-ITEM-STATUS  (how  hostile  is  it?),  and  its 


OBJECT-ITEM-AFFILIATION  (how  is  it  affiliated?). 
By  explicitly  encoding  the  attribute  information  con¬ 
tained  within  the  estimate,  any  node  can  understand 
what  attribute  information  is  available  in  the  estimate 
that  is  supplied  to  it. 

The  GaussianMixtureModel  class  encodes  the 
probability  distribution  used  to  encode  the  attribute 
values.  In  general,  the  uncertainty  in  one  attribute  is 
correlated  with  the  uncertainty  in  another  attribute. 
Therefore,  the  probability  must  be  maintained  within 
a  single,  joint  structure.  The  Estimate  class  maintains 
a  mapping  of  attributes  values  to  indices  within  the 
GaussianMixtureModel. 

The  final  class,  Time ,  specifies  the  time  at  which 
the  estimate  was  calculated.  Time  could  be  based, 
for  example,  on  the  last  time  that  an  observation  was 
received.  In  a  general  distributed  data  fusion  net¬ 
work  there  is  a  significant  issue  with  clock  synchro¬ 
nization  [15]  and  so  the  time  value  should,  itself,  main¬ 
tain  uncertainty.  For  this  initial  design  we  neglect  this 
uncertainty  but  note  that  data  fusion  algorithms  have 
been  developed  to  be  robust  to  measurements  with  un¬ 
known  but  bounded  time  delays  [16]. 

The  set  of  attributes  contained  within  an  Estimate 
can  change  through  time.  When  an  Attribute  is  to  be 
added  to  an  estimate,  a  new  instance  is  created  and 
registered  with  the  Estimate  using  the  addAttribute 
method.  This  method  assigns  a  new  index  to  the  at¬ 
tribute,  stores  its  value  in  the  attribute  list,  and  re¬ 
sizes  the  GaussianMixtureModel  to  include  the  new 
attribute.  When  an  Attribute  is  deleted  using  the  re- 
move Attribute,  it  is  removed  from  the  attribute  list, 
the  GaussianMixtureModel  is  resized  and  all  indices 
updated. 

5  Operations  Within  a  Single 
Node 

The  operation  within  a  node  consists  of  the  steps  out¬ 
lined  by  the  filtering  loop  shown  in  Figure  2:  the  es¬ 
timate  is  initialized,  predicted  and  updated.  However, 
these  steps  can  only  be  achieved  by  constructing  the 
appropriate  mathematical  models  and  using  them  to 
transform  the  uncertainty  associated  with  an  estimate. 
In  turn,  these  depend  on  the  structure  of  the  target  es¬ 
timate  and  the  capabilities  of  the  node. 

5.1  Constructing  the  Mathematical 
Models 

As  explained  above,  the  structure  of  the  target  state 
varies  both  through  time  and  across  the  network.  How¬ 
ever,  maintaining  an  exhaustive  list  of  all  possible  mod- 


Figure  6:  The  composable  process  model  is  a  set  of 
ProcessModelStrategy  objects.  Each  object  acts  on  an 
input  set  of  attributes  and  yields  an  output  set.  For 
those  attributes  without  an  explicit  model  (such  as  at¬ 
tribute  A6)  a  constant-value  mapping  is  provided. 

els  for  all  possible  structures  of  the  target  state  can 
become  prohibitively  expensive. 

5.1.1  Composable  Models 

We  overcome  these  difficulties  by  making  the  models 
composable.  The  principle  is  illustrated  in  Figure  6  for 
a  process  model.  The  process  model  takes  the  state 
estimate  at  a  time  step  k  and  generates  a  new  state  es¬ 
timate,  with  the  same  structure,  at  time  step  k+1.  The 
model  is  composed  of  a  set  of  ProcessModelStrategy  ob¬ 
jects.  Each  ProcessModelStrategy  takes  an  input  of  a 
set  of  attributes  and  yields  an  output  set  of  attributes. 
The  input  and  output  attribute  sets  for  a  ProcessMod¬ 
elStrategy  do  not  have  to  be  the  same.  For  example, 
the  ProcessModelStrategy  might  take  position  and  ve¬ 
locity  attribute  information  and  predict  a  new  position 
based  on  the  assumption  that  the  velocity  is  constant. 
Furthermore,  default  mappings  can  be  provided  if  none 
are  defined. 

The  same  approach  can  be  provided  composable  ob¬ 
servation  and  initialization  models. 

However,  to  compose  an  appropriate  ProcessModel, 
the  appropriate  strategy  components  must  be  chosen. 

5.1.2  Identifying  the  Correct  Model  Strategy 
Components 

The  choice  of  strategies  to  construct  the  process  model 
is  influenced  by  three  factors.  First,  the  collection  of 
strategies  are  chosen  such  that  all  of  the  required  at¬ 
tributes  are  updated.  Second,  it  is  possible  to  use  mod¬ 
els  of  different  fidelity  to  describe  the  behavior  of  the 


same  set  of  attributes.  For  example,  a  model  of  air¬ 
craft  motion  could  assume  constant  velocity  or  it  could 
use  complicated  aerodynamic  equations  to  predict  the 
aircraft’s  path.  Finally,  it  is  possible  to  have  conflicts 
—  two  ProcessModelStrategy  objects  could  attempt  to 
update  the  same  attribute  value.  Our  solution  is  to  use 
a  RideEngine  to  pick  and  choose  the  collection  used. 
Each  ModelStrategy  advertises  its  input  attributes  and 
output  attributes  that  define  its  inputs  and  outputs.  In 
addition,  each  strategy  can  advertise  something  about 
its  computational  costs  as  well.  The  RideEngine  then 
picks  and  chooses  the  combination  of  models  that  pro¬ 
vide  the  minimum  cost.  Our  current  strategy  for  avoid¬ 
ing  conflicts  is  to  use  the  value  from  the  strategy  object 
with  the  lowest  cost. 


5.2  Projecting  Uncertainty  Through 
the  Models 

The  previous  subsection  described  how  strategy  ob¬ 
jects  can  be  assembled  to  dynamically  build  models 
at  runtime.  However,  the  filtering  algorithm  prop¬ 
agates  estimates  with  uncertainties  associated  with 
them.  One  means  of  combining  the  two  together  is  to 
assume  that  ModelStrategy  objects  have  to,  for  exam¬ 
ple,  be  able  to  provide  Jacobians  of  their  computations 
to  be  used  in  an  extended  Kalman  Filter.  However,  in 
this  paper  we  advocate  the  use  of  numerical  techniques 
such  as  the  Unscented  Transform  [17]  or  particle  fil¬ 
ters  [18].  The  reason  for  using  these  techniques  is  that 
each  ModelStrategy  object  can  be  considered  to  be  a 
“black  box”:  it  takes  a  set  of  inputs  and  transform 
them,  but  no  knowledge  of  the  internal  structure  is  re¬ 
quired.  Furthermore,  using  a  set  of  discrete  samples, 
almost  any  mathematical  quantity  about  the  transfor¬ 
mation  can  be  calculated  and  thus  almost  any  data 
fusion  algorithm  can  be  used. 

The  numerical  scheme  is  implemented  using  the 
SampleRealization  class.  This  class  contains  a  set  of 
(weighted)  samples  drawn  from  the  Estimate  together 
with  an  Attribute  list  to  specify  the  indices  for  each 
attribute.  The  process  and  other  models  are  applied 
to  each  sample  in  turn  and  the  statistics  of  the  trans¬ 
formed  set  are  calculated. 

6  Interaction  Between  the 
Nodes 

Nodes  do  not  operate  in  isolation;  rather  they  function 
in  a  network  and  they  must  be  able  to  exchange  models 
and  data. 


Figure  7:  The  Advertisement  architecture. 

6.1  Negotiation  Between  the  Nodes 

Fusion  nodes  exchange  data  and  models  dynamically 
via  the  Remote  Method  Invocation  (RMI)  capabilities 
that  underly  the  Grid.  For  the  fusion  nodes  to  under¬ 
stand  what  data  and/or  models  need  to  be  exchanged, 
they  use  the  Grid  to  advertise  the  capabilities  of  the 
agent  and  query  the  LUS  to  ascertain  the  capabilities 
of  other  agents  and  services. 

When  a  fusion  node  starts  up,  the  first  thing  it  does 
is  try  to  register  with  an  instance  of  the  Look  UP  Ser¬ 
vice  (LUS)  on  the  CoABS  Grid.  If  the  node  cannot 
connect  to  an  instance  of  a  LUS,  the  fusion  node  has 
two  strategies  it  can  adopt.  The  first  is  that  the  node 
can  create  its  own  LUS.  As  explained  above,  multiple 
redundant  LUSs  can  run  on  the  same  network  without 
difficulty.  However,  some  nodes  (such  as  embedded 
sensors)  might  not  have  the  capability  to  start  their 
own  LUS.  In  this  case,  the  node  operates  in  a  stan¬ 
dalone  mode  and  polls  for  an  LUS  intermittently.  Once 
a  connection  has  been  made,  the  node  advertises  its  ca¬ 
pabilities  to  the  LUS.  It  also  downloads  from  the  LUS 
a  list  of  capabilities  of  all  of  the  nodes  within  its  range. 

Each  node  contains  a  rule  engine  that  is  written 
specifically  for  that  type  of  fusion  node.  This  rule  en¬ 
gine  contains  all  of  the  rules  needed  to  query  the  LUS 
for  other  nodes  that  it  can  communicate  with  and  have 
data  and/or  models  that  it  can  make  use  of  locally.  Al¬ 
ternatively,  it  could  request  that  another  node  in  the 
network  perform  a  calculation  for  it,  thus  offloading 
some  of  its  computational  burden. 

All  of  the  fusion  nodes  derive  their  advertisements 
from  the  Abstract  Advertisement  class  of  the  Grid,  from 
which  a  class  hierarchy  (Figure  7)  of  class  objects  is 
created  that  allows  various  types  of  meta  data  to  be 
posted. 

There  are  three  main  types  of  advertisements  that 


are  used  in  the  architecture:  AttributeAdvertisements, 
Database  Advertisements,  and  ModelAdvertisements. 
The  Model  Advertisement  is  further  developed  into 
advertisements  for  each  of  the  types  of  models  that  are 
used  by  the  fusion  algorithms.  These  advertisements 
allow  other  fusion  nodes  to  query  the  lookup  service  for 
raw  data,  new  models  for  processing  that  data,  and  to 
discover  what  types  of  databases  are  on  the  network 
for  querying. 

Each  fusion  node  locally  keeps  an  object  called  an 
AgentFinder.  The  AgentFinder  implements  the  Grid 
interface  ServiceListener.  The  ServiceListener  inter¬ 
face  provides,  via  an  event  based  mechanism,  the  abil¬ 
ity  to  discover  when  new  agents  register  or  deregister 
from  the  network,  and  when  the  advertisements  for  any 
agent  change.  The  AgentFinder  keeps  a  local  cache  of 
this  information  and  thus  avoids  the  need  for  addi¬ 
tional  queries  across  the  network.  From  its  rule  engine 
the  fusion  node  will  determine  the  best  way  to  request 
data  and  models  from  other  fusion  nodes  and  then  send 
out  a  request  message  requesting  either  data,  or  data 
and  models.  These  requests  are  stored  locally  by  a  fu¬ 
sion  node,  and  each  time  an  observation/measurement 
is  taken  by  one  of  its  sensors,  it  will  check  a  local  ta¬ 
ble  containing  these  requests  in  order  to  determine  if 
other  fusion  nodes  have  requested  the  data.  The  node 
will  determine  which  of  the  other  nodes  have  requested 
the  data,  publish  it,  and  if  a  model  was  also  requested 
that  will  be  included  with  the  data.  Through  RMI,  the 
other  fusion  node  will  not  have  to  have  that  model’s 
codebase  locally;  it  will  be  able  to  download  it  via  the 
capabilities  of  RMI  and  use  it  locally,  even  though  it 
did  not  originally  exist  locally  for  that  remote  node. 


6.2  Handling  Changes  in  Network 
Topology 

Over  time  the  network  topology  will  change  as  sensor 
nodes  are  added  and  removed.  All  agents  in  the  net¬ 
work  retain  some  knowledge  of  their  topology  through 
querying  of  the  LUS(s),  however  what  happens  if  no 
LUS  is  available?  A  LUS  could  be  viewed  as  a  point  of 
failure  thus  creating  a  more  centralised  network.  With¬ 
out  the  LUS,  the  organization  of  the  sensor  nodes  and 
distribution  of  their  states  would  be  very  difficult  to 
manage.  So  in  order  to  maintain  the  decentralized 
nature  of  the  network,  i.e.  it  has  no  central  node, 
the  architecture  must  be  robust  enough  to  detect  if 
an  agent  can  see  a  LUS,  and  in  the  case  of  the  LUS’s 
absence,  start  its  own  LUS.  Multiple  instances  of  LUSs 
can  share  information  across  the  network. 


6.3  Communication  Between  Sensor 
Nodes 

The  process  by  which  nodes  communicate  is  simple. 
They  already  maintain  local  stores  of  dynamically  up¬ 
dated  information  concerning  the  other  nodes  within 
their  topology,  from  which  a  node  can  choose  to  re¬ 
quest  data  and/or  algorithms  from  another  node.  A 
simple  rule  engine  is  used,  where  the  rules  for  use  of 
data  and  algorithms  is  known  a  priori.  The  rule  engine 
will  determine  the  following: 

•  What  data  from  the  remote  agent  is  readily  usable 
locally? 

•  What  data  from  the  remote  agent  used  in  conjunc¬ 
tion  with  local  data,  can  be  combined  to  create 
new  attributes  in  the  state  estimate?  For  example, 
agent  A  contains  estimates  on  range  of  a  target. 
Agent  B  contains  estimates  on  bearing  of  a  target. 
If  an  algorithm  for  combining  range  and  bearing 
estimates  that  returns  an  estimation  of  position  is 
available,  then  create  a  new  attribute  field  in  the 
local  state. 

•  What  algorithms  can  be  downloaded  from  the  re¬ 
mote  agent? 

•  If  a  sophisticated  algorithm  cannot  be  used  locally 
due  to  computational  constraints,  can  the  remote 
agent  perform  the  calculations? 

It  is  interesting  to  note  that  an  algorithm  that  is 
downloaded  in  order  to  give  new  capabilities  locally 
can  be  kept  locally  and  used  more  than  once.  For  in¬ 
stance,  a  sensor  that  detects  bearing  and  range  is  fused 
with  another  sensor’s  data  adding  a  position  attribute 
to  its  state.  The  sensor  didn’t  originally  possess  a  po¬ 
sition  algorithm;  it  was  provided  by  the  other  sensor 
and  downloaded  through  the  mobility  of  code  in  the 
Grid.  Should  the  two  agents  lose  network  connectiv¬ 
ity  to  each  other,  the  sensor  can  still  use  the  algorithm 
locally,  updating  the  position  attribute  as  new  observa¬ 
tions  on  range  and  bearing  are  obtained.  From  a  data 
fusion  stand-point,  this  estimate  would,  over  time,  di¬ 
verge  and  statistically  more  noise  would  enter  into  the 
calculation.  However,  should  the  two  nodes  come  into 
contact  again,  the  estimates  can  readily  be  updated, 
once  again. 

7  Example 

Consider  the  scenario  shown  in  Figure  8.  A  single 
target  is  being  tracked  by  two  sets  of  sensors:  an 
unattended  seismic  ground  sensor  (USGS)  and  an  un¬ 
manned  aerial  vehicle  (UAV).  The  target  is  moving 


Figure  8:  The  scenario. 


through  a  mountain  pass,  which  is  being  monitored  at 
first  only  by  the  USGS.  The  USGS  observes  seismic 
data  and,  from  this,  it  can  estimate  the  range  to  the 
target  and  its  spectrogram.  The  USGS  state  vector 
consists  of  the  following  states: 

USGS  =  (range,  spectrogram) 

This  state  is  updated  at  the  command  center  where 
a  UAV  is  dispatched  for  imagery  reconnaissance.  The 
UAV  carries  LIDAR  and  EO/IR  imagery  sensors.  It 
contains  attributes  for  imagery  both  infrared  and  vi¬ 
sual,  position  estimated  from  the  imagery  and  the  esti¬ 
mated  GPS  position  of  the  UAV,  and  a  range-to-target 
measured  in  kilometers  from  the  LIDAR.  At  first,  since 
the  USGS  is  within  the  mountain  pass,  until  the  UAV 
is  overhead  it  will  not  be  able  to  communicate  directly 
with  the  USGS.  Once  it  has  reached  the  pass,  and  is 
able  to  see  the  target  with  its  sensors,  the  UAV  will 
create  a  state  estimate  consisting  of  the  following  val¬ 
ues: 

UAV  =  (targetjxyz,  uavjxyz,  range,  irdmage, 
visuaLimage) 

Note  that  both  sensors  start  by  containing  only  the  at¬ 
tributes  that  their  respective  onboard  sensors  can  mea¬ 
sure.  Over  time,  as  the  two  nodes  continue  to  track  the 
target,  they  will  exchange  estimates  from  their  states. 
If  it  is  determined  that  data  from  one  node  can  be  used 
by  another  node,  they  will  exchange  that  data.  This 
may  cause  the  state  of  one  of  the  nodes  to  be  updated 
with  a  new  value  not  previously  contained  within  the 
state.  Referring  to  our  example,  the  UAV  contains 
the  attribute  for  position  within  its  state,  and  the  seis¬ 
mic  sensor  contains  a  range-to-target.  These  are  values 
that  can  readily  be  fused  together  to  create  new  esti¬ 
mates  of  position  and  range.  The  UAV  would  provide 
the  USGS  with  this  data,  and,  if  necessary,  a  model 
for  calculation.  At  this  point  the  seismic  sensor  would 
add  an  attribute  for  position  within  its  state  and  the 
state  would  become: 

USGS  =  (range,  spectrogram,  targetjxyz) 

At  the  same  time,  the  UAV’s  state  becomes  more  ac¬ 
curate  since  its  position  attribute  is  being  fused  with 


the  range  attribute  from  the  USGS.  Notice  that  these 
two  sensor  nodes  do  not  need  to  exchange  data  from 
the  imagery,  nor  signature  and  intensity  of  the  signal 
contained  within  the  spectrogram,  since  these  values 
are  not  readily  fusable  with  any  of  the  other  values 
contained  within  either  node. 

8  Conclusions 

This  paper  has  described  an  agent-based  architecture 
for  heterogeneous  distributed  data  fusion  algorithms. 
The  algorithm  generalizes  the  notion  of  a  target  state 
to  a  dynamically  adjustable  collection  of  attributes 
with  uncertain  values.  Mathematical  models  can  be 
constructed  in  real  time  on  the  basis  of  their  inputs 
and  outputs  and  their  computational  costs.  We  have 
discussed  how  advertisements  can  be  used  to  refine  the 
communication  between  agents  and  we  have  illustrated 
these  results  in  a  target  tracking  example. 

We  will  be  extending  this  architecture  in  two  re¬ 
spects.  First,  we  shall  integrate  our  architecture  us¬ 
ing  Fuselet  Technology  [5].  Second,  we  shall  extend 
the  representation  of  state  to  support  complicated  at¬ 
tribute  types  such  as  non- numeric,  discrete  quantities. 
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